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Abstract 

In  the  context  of  transforming  military  forces,  activities  in  Concept  Development  & 
Experimentation  (CD&E)  are  increasingly  seen  as  important  with  respect  to  the  realization  of 
needed  capabilities  in  areas  where  there  are  no  established  solutions.  Limited  funding  leads  to  a 
need  for  prioritization  of  proposed  experiments.  This  paper  describes  a  methodology  that  helps 
prioritize  experimental  activities  on  the  basis  of  operational  values,  costs  and  risk  profiles.  The 
methodology  has  been  developed  in  the  project  METEX  (METhod  for  Experimentation) 
conducted  by  Norwegian  Defence  Research  Establishment  (FFI)  and  Teleplan  AS  in  Norway. 

The  paper  describes  a  decision  support  tool  for  prioritization  of  experimental  activities  based 
on  operational  need,  operational  value  and  the  estimated  cost  and  risk.  For  assessing 
operational  value,  a  network  centric  representation  of  the  force  structure  is  applied.  The  use  of 
properties  and  gaps  in  this  representation  of  the  force  structure  is  crucial  in  the  methodology. 

The  paper  also  briefly  describes  a  web-based  process  framework,  in  which  the  assessment  tool 
is  incorporated.  The  web  based  framework  serves  as  a  guide  through  the  experimentation 
process. 

The  Norwegian  Armed  Forces  plan  to  implement  the  methodology  and  associated  process 
within  this  year  (2004)  to  support  the  CD&E  process. 


Introduction 

The  wide  spectrum  of  current  and  future  security  challenges  impose  new  requirements  on  our 
Armed  Forces.  As  a  result  there  is  a  need  for  adapting  the  defense  thinking,  planning  and 
defense  structures.  The  overall  goal  is  to  have  interoperable  and  usable  forces  available,  when 
they  are  needed,  where  they  are  needed,  and  with  the  capabilities  that  are  needed. 

Transformation  means  re-shaping  the  military  body  to  give  it  the  necessary  agility,  punch, 
technique  and  mobility  to  make  it  usable  in  present  and  future  security  scenarios. 

Transformation  implies  a  focus  on  developing  new  concepts,  how  to  do  things  better, 
experimenting  with  solutions  to  verify  or  falsify  hypotheses  regarding  operational  value,  and 
demonstrating  and  finalizing  some  of  these  solutions. 

Concept  Development  and  Experimentation  (CD&E)  is  often  looked  upon  as  a  vehicle  for 
moving  along  the  road  of  transformation.  A  properly  focused,  well  balanced,  and  expertly 
conducted  CD&E  program  is  paramount  to  take  full  advantage  of  the  opportunities  that 
information  age  concepts  and  technologies  offer. 

CD&E  activities  will  partially  emerge  from  top-down  initiatives  based  on  identification  of 
capability  gaps  relative  to  plans  for  needed  capabilities,  and  partially  on  bottom-up  initiatives 
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(shortfalls  and  “good  ideas”)  originating  from  communities  concerned  with  e.g.  operations, 
R&D  or  training. 

The  Joint  Operational  Headquarter  in  Norway  has  been  given  the  responsibility  for 
coordinating  all  national  operationally  related  experimentation,  and  due  to  limited  resources,  a 
framework  to  facilitate  prioritization  of  proposed  experiments  is  needed.  Experiments  are 
likely  to  have  different  cost  and  risk  profiles  which  should  be  weighted  against  the  potential 
operational  benefit  of  the  associated  concept. 

The  prioritization  framework  should  also  support  a  process,  which  balances  top  down 
initiatives  and  bottom  up  creativity.  Identification  of  capability  gaps  will  fonn  a  basis  for 
prioritizing  initiatives. 

This  paper  describes  a  framework  and  a  decision  support  tool  for  prioritization  of  experimental 
activities  based  on  operational  need  (capability  gap),  operational  value  (to  which  extent 
operational  capabilities  are  to  be  enhanced)  and  the  estimated  cost  and  risk  of  the  experiment. 
The  framework  is  currently  under  final  development  and  evaluation  by  the  Norwegian  Defence 
Research  Establishment  (FFI)  and  Teleplan  AS  in  a  project  denoted  METEX  (METhod  for 
Experimentation) . 

The  Norwegian  Anned  Forces  plan  to  implement  the  methodology  and  associated  process 
within  this  year  (2004)  to  support  the  future  CD&E  process. 


CD&E  process 

The  concept  development  process  begins  with  the  identification  of  a  concept  idea  and  runs 
through  concept  exploration,  further  development  and  refinement  until  completed  concept 
evaluation.  In  this  paper  the  CD&E  process  is  divided  into  the  four,  partially  overlapping 
processes:  identify,  develop,  experiment  and  evaluate. 

Figure  1  illustrates  the  CD&E  process  as  a  linear  four-step  process.  However,  in  real  life 
concepts  are  developed  through  several  parallel  and  iterative  processes.  For  instance,  the 
development  of  an  overarching  warfare  concept  may  be  accomplished  by  maturing  a  number  of 
identified  ideas  trough  parallel  iterations  of  the  succeeding  steps:  identify,  develop,  experiment 
and  evaluate. 

New  developments  typically  start  by  arranging  workshops  and  seminars  and  performing 
studies/analysis.  Then  it  may  be  followed  by  simulations  and  gaming,  before  carrying  out  more 
practical  tests,  such  as  CAX/CPX  or  field  exercises.  When  the  concept  has  reached  a 
satisfactory  level  of  maturity  the  development  process  is  completed  by  larger 
tests/demonstrations,  where  a  major  field  exercise  may  be  used  as  an  arena.  Final  experiments, 
which  are  arranged  as  demonstrations,  are  often  well  suited  for  addressing  a  wider  audience 
and  decision  makers. 

Identify 

In  this  phase  trends  and  ideas  are  explored  in  order  to  identify  new  concepts  worthwhile 
pursuing  for  further  exploration,  development  and  experimentation.  These  ideas  may  originate 
from  a  military  unit,  research  organization  or  industry  etc.,  and  should  have  a  potential  for 
providing  new  and/or  improved  operational  capabilities.  Overarching  “top-down”  concepts 
contributes  to  new  capabilities  in  the  long  term  where  as  “bottom-up”  concepts  often  are  aimed 
at  improving  existing  capabilities  or  fixing  shortfalls  in  the  short  term.  The  “top-down”  and 
“bottom-up”  developments  both  have  a  clear  justification  and  should  exist  side  by  side, 
complementing  each  other. 
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Figure  1:  Concept  Development  &  Experimentation  as  a  four-step  process 


Develop 

In  this  phase  the  concept  idea  is  explored  and  developed  through  workshops,  seminars,  studies 
and  analysis  to  prepare  for  practical  experimentation.  Potential  methods  and  techniques  for 
analyzing  the  concept  are  explored  and  compared  (Figure  2).  A  documentation  of  the  concept 
and  how  it  may  be  implemented  along  with  one  or  several  hypothesis  for  expected  operational 
value  are  developed. 

Experiment 

In  this  phase  the  complete  concept,  or  parts  of  it,  is  developed  through  experimentation. 
“Experiments”  in  this  context  may  be  simulations,  gaming,  small-scale  field  exercises  early  in 
the  development  process  or  a  larger/more  complex  demonstration  experiment  toward  the  end  of 
a  concept  development  (see  Figure  2). 

The  main  purpose  of  experimentation  is  to  verify  whether  a  hypothesis  or  the  goal  of  the 
concept  is  feasible  or  not.  An  experiment  that  failed  to  give  the  expected  results,  regardless  of 
the  experimentation  method,  might  in  principle  be  just  as  useful  as  an  experiment  that  did  give 
the  expected  results.  Normally,  the  conclusion  of  an  experiment  is  that  the  concept  is  either 
rejected  or  recommended  for  further  development.  Alternatively,  new  ideas  are  identified  from 
the  experiment.  Both  “top-down”  and  “bottom-up”  concepts  should  be  tied  to  an  overarching 
concept. 

Experiments  require  a  minimum  of  scientific  foundation  in  order  to  obtain  valid  results.  This 
may  lead  to  extensive  requirements  regarding  methodology,  metrics  and  documentation1. 

Evaluate 

In  the  evaluation  phase  the  results  and  findings  from  the  experiment  are  analyzed  and 
compared  with  findings  and  lessons  learned  from  previous  experiments.  Based  on  the  analysis 
and  increased  knowledge  on  the  concept,  recommendations  for  further  work  are  made. 

Framework  for  assessing  CD&E  activities 

A  framework  based  on  a  Network  Centric  Component  Model  is  used  as  a  basis  for  prioritizing 
CD&E  activities.  This  framework  has  been  established  in  order  to  assess  to  what  extent  an 
experiment  will  contribute  to  filling  a  gap  in  the  force  structure. 


1  US  DoD  Command  and  Control  Research  Program  -  Code  of  Best  Practice  for  Experimentation. 
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Figure  2:  Process  and  tools  for  maturing  and  developing  concepts 


The  Component  model  consists  of  four  building  blocks  as  shown  in  Figure  32 3: 

1)  A  decision  component,  which  consists  of  decision  support  and  “decision-making”  and  which 
deliver  decisions,  2)  A  sensor  component,  which  delivers  data  and  information  from  sensors,  3) 
An  effector  component,  which  delivers  “effects”  in  operations  (both  effects  obtained  directly 
and  indirectly  from  weapons  and  other  effects  that  is  important  for  the  conduct  of  operations) 
and  4)  An  information  infrastructure  (INI),  which  delivers  connectivity  and  distribution 
capacity  for  data  and  information.  The  INI  focus  is  on  communication  infrastructure  and 
infrastructure  management. 


Figure  3:  Component  model  used  for  representing  the  force  structure 


2  Based  on  ideas  from  UK  Command  and  Battlespace  Management  and  Canadian  Forces  Experimentation  Centre 
(Plan  Pegasus). 

3  This  model  is  based  on  a  model  developed  in  the  project  “Digitization  of  the  battlefield”  (SLADI)  at  the 
Norwegian  Defence  Research  Establishment. 
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The  decision,  sensor  and  effector  components  are  divided  into  sub-areas  covering  different 
command  levels  when  it  comes  to  the  decision  component  and  different  operational 
environments  for  the  effector  and  sensor  component,  see  Figure  4.4 

The  force  structure  shall  fulfill  certain  properties.  These  properties  must  be  chosen  so  that  they 
are  relevant  with  respect  to  possible  missions  for  the  military  units.  They  should  be  determined 
on  the  basis  of  top  level  prioritizations  for  the  force  structure.  Some  properties  are  common  for 
all  components  and  some  are  specific  with  respect  to  certain  components: 

•  Common  properties  (for  all  components) 

-  Robustness,  determined  by  physical  and  electromagnetic  survivability,  security 
and  logistical  aspects. 

-  Interoperability,  detennined  by  the  ability  to  operate  together  and  coordinated 
with  other  components  (SOP,  language,  organization,  standardization, 
technology,  doctrine,  culture). 

-  Speed  and  flexibility,  detennined  by  the  range  of  possible  mission  types  and 
operational  tempo. 

-  Responsiveness  and  strategic  deployability,  determined  by  the  ability  to  be 
deployed  and  redeployed  in  a  strategically  context. 

•  Component  specific  properties 

-  Decision  effectiveness  for  the  decision  component,  detennined  by  the  ability  for 
effective  decision-making.  This  is  determined  by  the  ability  to  establish  a 
relevant  situation  picture  (including  tools  supporting  presentation,  analysis  and 
simulations),  the  knowledge  and  experience  for  decision  makers  and  staff 
personnel  and  the  existing  culture. 

-  Coverage  and  data  quality  for  the  sensor  component,  detennined  by  the  ability 
to  catch  the  relevant  amount  of  data  and  represent  these  data  correctly. 

-  Effect  for  the  effector  component,  determined  by  the  ability  to  impact  targets 
and  obtain  other  desired  effects. 

-  Connectivity  and  distribution  capacity  for  the  INI  determined  by  the  number  of 
connection  points  (mobile  or  fixed)  and  transmission  capability. 

Figure  4  shows  how  gaps  in  the  force  structure  can  be  displayed.  Red  indicates  critical  gaps, 
yellow  indicates  substantial  gaps  and  green  represents  satisfactory  status.  The  colors  are  given 
by  the  current  as-is  situation.  The  gaps  are  given  at  a  fairly  aggregate  level,  but  are  supposed  to 
be  satisfactory  with  respect  to  prioritization  of  experimental  activities  at  an  overarching  level 
(operational/ strategic)5 . 


4  The  operational  environments  are  interpreted  as  the  area  in  which  the  component  (sensor,  effector)  has  its 
impact.  For  example,  an  elevated  sensor  might  see  into  the  land  environment,  and  if  it  does,  it  contributes  to 
sensor  coverage  in  that  environment  (land). 

5  For  prioritization  of  activities  at  a  lower  level,  for  instance  in  the  different  services,  the  structure  must  be 
represented  is  a  somewhat  different  manner;  Separate  matrixes  should  be  developed. 
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Figure  4:  Exemplified  (not  real)  representation  of  gaps  in  the  component  based  force 
structure  with  respect  to  properties.  Red  indicates  critical  gaps,  yellow  indicates 
substantial  gaps  and  green  represents  satisfactory  status. 


Decision  support  tool 

METEX  has  developed  a  decision  support  tool  based  on  the  above  framework  that  will  be  used 
when  prioritizing  experiments.  The  tool  gives  important  indications  of  whether  or  not  an  actual 
experiment  should  be  conducted.  It  should  be  noted,  however,  that  the  value  of  the  tool  itself 
increases  as  more  and  more  experiments  are  assessed;  it  is  in  the  comparison  of  different 
experiments  that  the  value  of  the  tool  is  demonstrated. 

The  tool  consists  of  four  modules  supporting  each  of  the  following  tasks: 

1)  Assessing  the  operational  value  (benefit)  of  the  experiment. 

2)  Calculating  the  costs  of  the  experiment. 

3)  Calculating  the  benefit/cost  ratio  of  the  experiment. 

4)  Assessing  the  uncertainties  associated  with  the  conduct  of  the  experiment. 

First,  associated  with  task  1),  the  operational  value  of  an  experiment  is  evaluated  due  to  the 
following  procedure  where  the  matrix  in  Figure  5  is  applied: 

a)  Find  the  points  in  the  matrix  given  by  components  and  associated  properties  related  to  the 
experiment 

b)  Set  a  score  in  the  blue  frame  (Figure  5)  next  to  those  points,  reflecting  to  what  extent  the 
gap  are  addressed.  A  score  set  to  1,  reflects  some  influence  on  the  gap,  and  a  maximum 
score  at  3  reflects  heavy  influence  of  the  experiment  on  that  gap. 

c)  These  scores  are  multiplied  with  the  respective  gaps  ( 1  for  green,  2  for  yellow  and  3  for 
red)  and  the  products  are  summed  up  for  each  component. 
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d)  The  operational  value  of  the  experiment  is  calculated  as  a  sum  of  all  total  scores  for  all  the 
components  (see  Figure  5)6. 

By  default  there  is  always  “gaps”  (set  to  3  in  the  matrix)  in  the  property  “cost/effectiveness”, 
meaning  that  improvements  in  cost/effectiveness,  without  necessary  influencing  the  other 
properties,  will  always  be  considered  and  prioritized  accordingly. 

Associated  with  task  2)  there  is  a  module  in  the  support  tool  that  helps  calculate  the  cost  of  an 
experiment.  This  module  helps  calculate  costs  associated  with  manpower,  equipment,  suppliers 
and  administration. 

Task  3)  is  to  calculate  the  benefit/cost  ratio  of  the  experiment  by  taking  the  numbers  for  benefit 
and  cost  and  place  those  in  a  diagram  for  comparison  with  other  experiments  (Figure  6).  The 
number  defined  by  dividing  the  score  from  task  1)  with  the  cost  from  task  2)  is  also  calculated, 
as  in  Figure  6,  where  the  actual  experiment  (in  black)  is  compared  to  other  experiments 
(shaded). 

Task  4  is  to  assess  the  uncertainty  for  the  conduct  of  the  experiment,  for  instance  in  tenns  such 
as: 

a)  Probability  for  not  getting  access  to  equipment,  infrastructure  and  personnel. 

b)  Probability  that  uncontrollable  factors  (such  as  weather)  will  influence  the  validity  of  the 
experiment  in  a  negative  way. 

c)  Probability  that  external  suppliers  cannot  deliver  to  estimated  cost  and  quality. 

In  that  way  a  “risk  profile”  for  the  feasibility  of  the  experiment  can  be  obtained,  and  this  profile 
is  also  an  important  factor  in  the  assessment  of  the  experiment. 

Going  through  the  four  tasks  for  each  experiment  in  a  portfolio  of  experiments  will  thus  give  a 
basis  for  prioritization  of  which  experiments  to  conduct  (fund). 


6  There  is  also  an  option  for  addressing  the  significance  (weight)  of  a  property.  In  Figure  5  this  significance  is  set 
to  the  (arbitrary)  value  of  3  for  all  properties. 
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Figure  5:  An  experiment  is  evaluated  and  given  different  scores  with  respect  to  how  the  results 

may  influence  various  properties  and  components. 
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Figure  6:  Presentation  of  benefit  and  cost  and  the  calculation  of  the  ratio  of  benefit  of  cost. 
The  actual  experiment  under  consideration  (black)  is  compared  to  other  experiments 

(shaded). 
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Web  based  process  framework 

Supporting  the  CD&E  process  and  the  prioritization  of  experiments  a  web  tool  has  been 
developed.  This  tool  guides  and  supports  the  decision  maker  through  the  assessment  process. 
The  start  page  of  this  web  tool  is  shown  in  Figure  7. 
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-METEX  CD&E  intro  (ppt) 
-Terms  and  definitions 
MFU03: 

-Concept  for  network  enabjj 
use  of  military  powe^ 
-Command  concept  in  NC’ 
-Final  report  DP2  and  3  - 
NCW  study  (ppt) 

Links: 

-International  links 


Area  for  supporting 
information  and 
useful  links 


METEX  Web 


05.  October  2003 
Do  you  have  ideas  for 
improvement?  Send  to 

Webmaster 


Area  for  direct  access  to 
supporting  documents  in 
original  format  (word,  excel 
etc.)  for  easy  printing  and 


Concept  Development  and' — 


Experimentation 


JIETEX  news: 


Available  area  for 
dynamic  information, 
news  etc. 


i-6  November  03 

Project  METEX  exhibits  at  NATO  CD&E  conf|j 
Prague 


05  .  October  03 

First  draft  version  of  the  METEX  process  released  on 
Web. 


©  Webdesign  Teleplan  AS 


•g]  Local  intranet 


Figure  7:  The  start  page  of  the  METEX  process  web 


Discussion 

Some  areas  described  above  need  some  further  elaboration  and  discussion.  This  is  mainly 
related  to  the  practical  use  of  the  models  and  tools.  In  particular,  some  considerations 
concerning  the  practical  use  of  the  assessment  model  should  be  appropriate. 

The  assessment  model  divides  the  military  structure  into  components.  The  sensor  and  effector 
components  have  impacts  in  different  operational  environments.  This  is  a  point  that  has  to  be 
stressed  for  the  decision  makers  using  the  model  and  for  the  experimental  communities 
proposing  experiments.  For  example,  aircrafts  in  the  role  of  close  air  support  or  as  platforms 
delivering  precision  guided  munitions  against  targets  in  the  land  enviromnent  contributes  to  the 
effector  in  the  land  environment. 

The  partition  into  operational  and  tactical  levels  for  the  decision  component  is  a  matter  of 
temporal  practical  convenience,  because  initially  we  wanted  to  associate  various  headquarters 
and  command  posts  with  the  decision  component.  This  partition  should  be  seen  in  a  temporary 
context,  because  such  a  partition  does  not  seem  to  support  a  network  centric  approach. 
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Also,  the  interfaces  between  components  should  be  defined  as  rigorously  as  possible  in  the 
tool.  For  instance,  the  interface  between  the  INI  and  the  decision  component  must  be  defined 
with  respect  to  infonnation  systems  and  infonnation  management. 

It  is  also  important  that  those  who  suggest  experiments  describe  them  so  that  it  is  easy  to  map 
the  impact  of  the  experiment  into  the  assessment  model.  In  the  process  web,  there  are  templates 
that  facilitate  this  description,  so  the  impact  on  components  and  properties  are  correctly 
addressed. 

The  assessment  model  is  primarily  a  decision  support  tool.  In  order  to  make  qualified 
decisions,  there  must  be  a  process  of  interaction  between  those  who  suggest  experiments  and 
the  decision  maker.  The  process  web  also  facilitates  this  interaction.  Using  the  process  web 
helps  the  experimenter  improve  the  design  of  an  experiment,  increase  the  operational  value  and 
reduce  the  costs. 
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Guiding  Experimentation  Efforts 


Chief  Scientist  Johan  Aas,  FFI 
DrTrond  Bergene,  Teleplaj>A€ 


in  Support  of  Transformation 


Transformation  and  CD&E 


CD&E 

•  Concept  Development  and  Experimentation  (CD&E)  is  paramount 
to  the  success  of  transformation  efforts. 

•  Top-down  initiatives:  Generation  of  CD&E  activities  based  on 
identification  of  capability  gaps. 

•  Bottom-up  initiatives  (shortfalls  and  “good  ideas”):  Generation  of 
CD&E  activities  from  communities  concerned  with  e.g.  operations, 
R&D  or  training. 

•  Prioritization  of  proposed  experiments  is  essential. 


CD&E  in  Norway 


•  The  Joint  Operational  Headquarter  in  Norway  is  responsible  for 
coordinating  all  national  operationally  related  CD&E 

-  This  differs  from  some  other  countries,  where  dedicated 
centers  have  been  given  the  responsibility  for  CD&E. 


METEX  (METhod  for  Experimentation):  The  framework  is  currently 
under  final  development  and  evaluation  by  the  Norwegian  Defence 
Research  Establishment  (FFI)  and  Teleplan  AS. 


Presentation  Outline 


•  The  CD&E  Process. 

•  A  framework  and  a  decision  support  tool  for  prioritizing 
experimental  activities. 

•  A  Web-based  process  framework  guiding  prioritization  of  CD&E 
activities. 


The  CD&E  Process 


Tools  Supporting  Maturation 
and  Development  of  Concepts* 


Immature 


Mature 

- > 


Decreasing  number  of  iterations 

Increasing  complexity  and  cost 
- » 


*  Based  on  ideas  from  UK  Command  and  Battlespace  Management  and 
Canadian  Forces  Experimentation  Centre  (Plan  Pegasus). 


The  CD&E  Process 


The  CD&E  process  is  divided  into  4  sub-processes: 


The  CD&E  Process:  Identify 


Identify 


CD&E 

Strategy 

_ 

New 

trends 


Potential 

concepts 

_ P 

•Research 

•Seminars 

•Etc 


The  CD&E  Process:  Develop 


The  CD&E  Process:  Experiment 


*:  US  DoD  Command  and  Control  Research  Program  - 
Code  of  Best  Practice  for  Experimentation. 


The  CD&E  Process:  Evaluate 


The  CD&E  Process 


A  framework  and  a  decision  support  tool 
for  prioritization  of  experimental  activities 


Network  Centric  Component  Model  (NCCM) 


The  NCCM  has  been  developed  in  connection  with  the  recent 
national  Defense  Requirement  Review: 


A  decision  component,  which  consists 
of  decision  support  and  “decision- 
making”  and  which  deliver  decisions. 

A  sensor  component,  which  delivers 
data  and  information  from  sensors. 


An  effector  component,  which  delivers 
“effects”  in  operations. 

An  information  infrastructure  (INI), 

which  delivers  connectivity  and 
distribution  capacity  for  data  and 
information. 


Components  Properties(l) 


Properties  are  used  to  characterize  NCCM  components. 


Information  infrastructure 


Common  properties  (for  all  components) 

-  Robustness 

-  Interoperability 

-  Speed  and  flexibility 

-  Responsiveness  and  strategic 
deployability 


\ 


/ 


Sensor 

component 


Effector 

component 


Components  Properties  (2) 


Component  specific  properties 

-  Decision  effectiveness  for  the  decision 
component 


-  Coverage  and  data  quality  for  the 
sensor  component 


Sensor 

component 


-  Effect  for  the  effector  component 


Effector 

component 


-  Connectivity  and  distribution  capacity 
for  the  INI 


(  -N 

Information  infrastructure 

V _ ) 


Gaps  in  the  Force  Structure  Based  on 
Components  and  Properties 


Space 

1 

2 

Information  infrastructure,  INI 

Connectivity  and  distribution  capacity 

INI 

2 

1 

2 

2 

•  Red:  Critical  gaps 

•  Yellow:  Substantial  gaps  The  colors  are  given  by  the  current  situation 

•  Green:  Satisfactory  status. 
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Assessing  the  Operational  Value  of  a  Concept 


[Total  score  84 


Decision  Support  Tool  Modules 


•  The  tool  consists  of  four  modules  supporting  each  of  the 
following  tasks: 

-  Assessing  the  operational  value  (benefit)  of  the  experiment. 

-  Calculating  the  costs  of  the  experiment. 

-  Calculating  the  benefit/cost  ratio  of  the  experiment. 

-  Assessing  the  uncertainties  associated  with  conducting  the 
experiment. 

•  Going  through  the  tasks  for  each  experiment  in  a  portfolio  of 
experiments  will  give  a  basis  for  prioritization  of  which 
experiments  to  conduct  (fund). 


A  Web-based  Process  Framework  Guiding 
Prioritization  of  CD&E  Activities 


METEX  Web  Front  Page 
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Sub-Processes  are  Accessed 
by  Point-and-Click 
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The  “Qualify  Idea”  Sub-Process  Describes  Tasks 
and  Provides  Check-Lists,  Templates,  etc 


2-1  Qualify  idea 


Final  Remarks 


The  methodology  is  currently  under  final  development  and 
evaluation 

•  An  early  release  is  already  in  use  by  the  Armed  Forces 


Final  observations 

•  In  order  to  make  qualified  decisions,  interaction  between 
decision  makers  and  experimenters  is  critical 

•  The  process  web  facilitates  this  interaction,  helps  the 
experimenter  improve  the  experiment  design,  increases  cost- 
benefit 

•  The  quality  of  an  actual  experiment  is  improved 


